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1 . INTRODUCTION 


During Phase III of the Large Area Crop Inventory Experiment (LACIE), 24 U.S. 
and 10 Canadian intensive test sites were scheduled for processing on the Inter- 
active Multispectral Image Analysis System, Model 100 (Image 100) using the 
Procedure 1 hybrid classification system. This report describes an evaluation 
of the results obtained. 

In this study the only segments analyzed were those with (1) satisfactory 
classifications and (2) complete classification and ground-truth data. This 
eliminated 22 of the 34 segments: 4 segments were not classified, 6 segments 

were eliminated because of unsatisfactory classifications, and 12 segments 
were eliminated because of inadequate classification or ground-truth data. 

This left 12 segments with 13 estimates. (Segment 1968 is a mixed-wheat seg- 
ment, so estimates for this segment are available for both spring and winter 
wheat. ) 

Originally, it was intended to investigate both labeling accuracy and propor- 
tion estimation accuracy. However, there were not enough dot-labeling data 
available, so this part of the investigation was abandoned. 

2. METHOD 

In the procedures used on the Image 100, the analyst could give proportion 
estimates for winter wheat, winter grains, spring wheat, or spring grains. 
However, these could not be compared directly with a corresponding ground- 
truth proportion, since the ground truth did not cover the whole segment. 
Therefore, an analyst estimate of the proportion in the ground-truth area was 
obtained and compared with the ground-truth value. This was done in the 
following manner. 

The image, classification map, and a map of the ground-truth area were read 
into the Image 100. Using existing hardware and software, the number of 
pixels within the ground-truth area and the number of these pixels which were 



classified as wheat or grains were counted. These pixel counts, after delet- 
ing those which had been designated other (00) pixels, were used to calculate 
the "uncorrected" proportion estimate for the ground-truth area. The bias 
correction alpha table, as stored in the Image 100, was then used to obtain a 
bias-corrected estimate for the ground-truth area. 

3. RESULTS 

The results of the evaluation are shown in table 1. Each segment is identi- 
fied in the table together with the acquisition used for the estimate, the 
type of estimate (i.e., spring wheat, winter wheat, spring grains, and winter 
grains), and the Robertson biostage as determined from the adjustable crop 
calendar for the latest imagery used. In each case, the uncorrected propor- 
tion estimate, X, and the bias-corrected proportion estimate, which are 
expressed as percentages, are given for the entire sequent. The corresponding 
quantities Y and for the ground-truth area within the segment are given in 
the next two columns. Since some of the estimates passed to the Crop Assess- 
ment Subsystem (CAS) were not corrected for bias, each estimate passed is 
identified by a superscript b. The ground-truth proportion, Y, which was 
determined from field reports, is also given, along with the difference, 

D = Yj, - Y, which is the proportion error for the ground-truth area. 

LACIF segment 1973 (Whitman County, Washington) had the largest difference 
value, D = -33.8. Labeling data were available for this classification, and 
they showed that all labeling errors were for wheat called nonwheat. Out of 
98 labeled type 1 and type 2 dots, approximately 25 percent were wheat dots 
which were labeled nonwheat. Note that 90 percent of the cultivated crops 
within the ground-truth area (excluding pasture and summer fallow) were 
winter wheat.* 


^Ground truth was av'aTTable for approximately one-tTiTrT^" fFTT^segment.^^l^^ 
area contained 53.5 percent winter wheat, 3.9 percent spring barley, 2.4 per- 
cent dry peas, and 40.1 percent fallow and pasture. 
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The largest overestimate (17.2 percent) occurred for LACIE segment 1992. 

This segment had only 2.3 percent spring wheat within the ground-truth area, 
but the large amount of barley raised the total spring grains to 40 percent. 
The high wheat estimate was caused primarily by the Inclusion of a large 
portion of barley in the wheat estimate. It Is particularly difficult to 
separate spring wheat and spring barley without an acquisition showing a 
difference in crop development such as occurs near the ripe stage of barley. 

The estimate for LACIE segment 1975 probably Is better than Is Indicated by 
the table 1 data. Examination of the imagery and the classification map 
indicates that the wheat proportion probably was much higher in the area out- 
side rather than inside the ground-truth area. The bias correction factors, 
which were calculated for the entire segment, probably are valid for the 
segment; however, because the wheat proportions are significantly different 
in the ground- truth area, these bias correction factors are not appropriate. 
Even so, the bias correction was applied to the ground-truth area for con- 
sistency. (See table 1.) 

Figure 1 shows a plot of the proportion errors as a function of the ground- 
truth proportions for all 13 estimates. It will be seen that the errors are 
overestimates for segments with low wheat proportions and that the errors are 
underestimates for segments with high wheat proportions. This behavior is 
similat" to that observed for blind sites (ref. 1) during LACIE Phase I and 
Phase II. 

With the exception of LACIE segment 1973, which was discussed earlier, fig- 
ure 1 shows that the largest proportion errors occurred in spring wheat and 
spring grains. Of the five spring wheat and spring grains estimates, two 
(segments 1958 and 1992) had very large overestimates of spring wheat in 
segments with very large proportions of barley and oats (46.0 and 38.6 per- 
cent, respectively). 
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Bias -corrected estimate minus ground truth {Y^ - Y), percent 






4. CONCLUSIONS 


Insufficient data are available to provide a clear picture of the accuracy of 

the Image 100 classifications, but some specific observations are valid. 

a. Labeling of wheat rather than total grains, particularly with only one 
acquisition, led to significant overestimates in some segments. 

b. The Image 100 software and procedures were written to facilitate classifi- 
cation of the LACIE segments but were not designed to record data for 
later accuracy assessment. A much better evaluation would have been pos- 
sible if accuracy assessment data had been collected following each 
satisfactory classification. 

c. A more thorough quality assurance check on the Image 100 analysis probably 
would have prevented reporting to CAS some of the segments (such as 1973 
and 1992) which had large estimation errors. 
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